Skip to content

Proposal: Rewrite SRA_RUNINFO_TO_FTP as native process #337

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
Parent: Added local.config
wants to merge 1 commit into
base: dev
Choose a base branch
from
Open

Proposal: Rewrite SRA_RUNINFO_TO_FTP as native process #337

wants to merge 1 commit into from

Conversation

@bentsherman
Copy link

Just a POC to demonstrate how to rewrite a simple process as an exec process.

The SRA_RUNINFO_TO_FTP process just transforms a metadata file and doesn't make any network requests, so it can be written purely in Nextflow pretty easily. That way you don't have to spin up a Python job just to do some metadata wrangling.

I think most people don't realize you can do this, due to the (historically) poor developer experience and lack of standard library documentation. Between the new strict syntax and upcoming standard library docs, I'm hoping people might find this approach more appealing.

A further optimization would be to not even write the samplesheet to a TSV file -- just emit the samplesheet as a list of maps.

@nf-core-bot
Copy link
Member

Warning

Newer version of the nf-core template is available.

Your pipeline is using an old version of the nf-core template: 3.0.2.
Please update your pipeline to the latest version.

For more documentation on how to update your pipeline, please see the nf-core documentation and Synchronisation documentation.

@github-actions
Copy link

nf-core pipelines lint overall result: Passed ✅

Posted for pipeline commit 20a3ead

+| ✅ 165 tests passed       |+
#| ❔  15 tests were ignored |#

❔ Tests ignored:

  • files_exist - File is ignored: .github/workflows/awsfulltest.yml
  • files_exist - File is ignored: .github/workflows/awstest.yml
  • files_exist - File is ignored: assets/multiqc_config.yml
  • files_exist - File is ignored: conf/igenomes.config
  • files_exist - File is ignored: conf/igenomes_ignored.config
  • files_exist - File is ignored: conf/modules.config
  • files_unchanged - File ignored due to lint config: .github/PULL_REQUEST_TEMPLATE.md
  • files_unchanged - File ignored due to lint config: assets/sendmail_template.txt
  • files_unchanged - File ignored due to lint config: assets/nf-core-fetchngs_logo_light.png
  • files_unchanged - File ignored due to lint config: docs/images/nf-core-fetchngs_logo_light.png
  • files_unchanged - File ignored due to lint config: docs/images/nf-core-fetchngs_logo_dark.png
  • actions_ci - actions_ci
  • actions_awstest - 'awstest.yml' workflow not found: /home/runner/work/fetchngs/fetchngs/.github/workflows/awstest.yml
  • multiqc_config - multiqc_config
  • modules_config - modules_config

✅ Tests passed:

Run details

  • nf-core/tools version 3.0.2
  • Run at 2025-02-21 00:20:03

@edmundmiller edmundmiller added this to the 1.13.0 milestone May 19, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

1.13.0

Development

Successfully merging this pull request may close these issues.

3 participants

@bentsherman @nf-core-bot @edmundmiller